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Abstract 

Many ELT examinations have gone online in the last few years and a large number of educational institutions have 
also started considering the possibility of implementing their own tests. This paper deals with the training of a group 
of 24 ELT teachers in the Region of Valencia (Spain). In 2007, the Ministry of Education provided funds to 
determine whether it would be possible to implement an online University Entrance Examination (RA.U.) in Spain 
at the national level. The project was to address three main areas: the technology, the students and the teachers. In 
relation to the teachers, the focus of this paper, the project was to investigate whether there were any changes in 
their routines, teaching methodology and attitudes towards assessment using technology. This brief study focuses on 
one of the last tasks in the preparation stage for implementation of the computerized exam, and it is intended to 
predict and observe the teachers’ reactions towards computer assisted language testing, the new test and the new test 
design. The findings shed light on the teachers’ internal changes and their changes in attitude throughout the process. 
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1. Introduction 

Computer assisted language testing (CALT) has influenced the way in which teachers approach language testing 
(Chapelle & Douglas, 2006). Many ELT tests have gone online in the last few years and a large number of 
institutions have also started considering the possibility of implementing their own online tests. This paper deals 
with the training of a group of 24 ELT teachers in the Region of Valencia in Spain. In 2007, the Ministry of 
Education granted financial support to the CAMILLE group of the Polytechnic University of Valencia to determine 
whether it would be possible to implement an online University Entrance Examination (In Spanish, prueba de 
acceso universitario\ henceforth, P.A.U.) in Spain at the national level. The project was address three main issues: 
the technology, the students and the teachers. In relation to the teachers, the focus of the present paper, the project 
was to investigate whether there were any changes in their routines, teaching methodology and attitudes towards 
assessment using technology. This brief study focuses on one of the last tasks in the preparation stage for 
implementation of the computer based exam, a teacher training course, and it is intended to observe and report the 
teachers’ reactions towards CALT, the new test and the new test design. The findings shed light on their internal 
changes and their changes in attitude throughout the process. The paper will not address the benefits of training on 
their attitudes towards technology but, instead, offer a general overview of the main difficulties found in the training 
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period. In order to determine them, the research team observed three main areas: problems related to test awareness, 
the priorities in the new test tasks, and the teachers’ difficulties in managing ICT. 

2. Background: the Place of the P.A.U. Examination in English Language Teaching 

The P.A.U. is by far one of the most important high-stakes examinations for many students in Spain. About 175,000 
students take it every year. Despite its importance, however, there seem to be few studies in relation to how teachers 
approach preparation of their students for the test (the so called washback effect) (Rea & Scott, 2007). 'Washback' 
denotes the influence of an examination on the teaching process (Alderson & Wall, 1992; Messick, 1996; Wall, 2000; 
Cheng, 2004; Hausenberg, 2006). Nevertheless, a few research notes and descriptive papers do assert that teachers 
are mostly influenced by the test tasks and their own experience. In practice, those tasks that are not included in a 
test usually receive little if any attention in the classroom activity (Yin, 2010). Consequently, students lack 
preparation in such skills as speaking (which is not included on the exam), while their knowledge of language use 
can be higher than their actual communicative skills (Koeppen et ah, 2008). The two most neglected skills are 
listening and speaking but the most susceptible to washback activities are grammar and translation (Wen-jun, 2010), 
so in a new test with a greater emphasis on speaking, the test would not be a major force in driving instruction 
(Munoz & Alvarez, 2009; Shih, 2009). 

Additionally, English teachers in Spain have not developed a special interest in the new information technologies, 
and they hardly use computers or language laboratories for instruction. Besides, although most students of 
translation, language education and philology in Spain (and possibly in the rest of Europe) will probably spend 
many years (if not to say their whole careers) in teaching —in which assessment is a crucial part of their job—, they 
receive almost no preparation for assessment in their university lessons (Stiggins & Conklin, 2002). Thus, teachers 
are not aware of the different factors involved in language testing (Cheng et al., 2008), and at the same time they 
need to be trained in this area (Baartman et al., 2007) as well as in the implications of high-stakes testing beyond 
their students’ need to obtain the highest score possible (Luxia, 2007). Consequently, in this “post-methodology” era 
(Kumaravadivelu, 2002), many instructors tend to use a limited number of tasks which may go against the idea of 
communicative use of language in education as both a means and a target (Savignon, 2007), even when doing so 
may go against their own beliefs (Saif, 2006; Turner, 2006) and thus lead to negative washback. Teachers can find 
abundant literature in the market to improve and focus their teaching, but Spanish teachers seem to rely more on 
their own intuition than on the professional literature (Garcia Laborda, 2001). In fact, there is dichotomy between 
what teachers think will help their students and what they actually do in class (Adir et al., 2006). 

Teachers spend a great deal of their time preparing their students for three main types of tasks: filling in exercises 
related to language use, reading small texts (up to 500 words) and writing short essays (usually around 200-250 
words). However, according to Garcia Laborda (2007), computer-based examinations allow for the inclusion of a 
large number of different tasks including receptive and productive oral tasks that can be more realistic than on a pen 
and paper test (a condition suggested by Wiggins, 1998). With all of the above considerations in mind, a training 
course for RA.U. teachers should include three main areas: preparation for language testing awareness, methodology 
for oral tasks, and technology use. These areas were also the main types of constraints that the researchers found in 
the current study. 

3. The Study 

3.1 Data Collection 

Twenty-four teachers were observed during a teacher training course in order to gather the data for this study. This 
course dealt with teachers training for the pilot Spanish University Entrance Examination. Specifically, their 
reactions and ways of approaching the tasks were observed and annotated. Afterwards, a 10 item test was 
administered to them on the computer. Their qualitative responses were grouped and analyzed according to the 
following topics: test awareness, difficulties in methodology, and constraints in the use of technology. 

3.2 Problems in Language Test Awareness 

The traditional approach to the RA.U. orientation for teachers in Spain, which is also currently used by the raters 
themselves (Amengual Pizarro, 2009), is descriptive. Teachers are occasionally given a 2-4 hour workshop at a 
university in which the test administrators simply explain the format of the test and the rating criteria. The criteria 
are then normally implemented by the teachers when they give examinations in the classroom. In the training course 
for this study, the teachers were asked to revise their assessment procedures and then use what they had learned in 
the training sessions to address some specific case studies. As a result, we observed the following situations among 
the 24 teachers: 
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1) Testing is not considered a fundamental part of the teaching process. The teachers thought that it was enough 
merely to use the same or similar exercises to those found in the students’ workbooks or those included in the 
teacher’s book to prepare a good test. 

2) Assessment is hardly ever seen as a process. Instead, it is viewed as a set of punctual acts of students’ providing 
evidence of their language knowledge and teachers’ giving scores. Thus, the principles of measurement and impact 
on the daily teaching process are usually ignored. 

3) The teachers limit the assessment process to exams and, at times, to subjective observations, so decision-making 
can be highly influenced by subjective feelings which may not be consistent with the students’ real potential and/or 
their language knowledge (and use). 

4) The teachers believe that assessment influences motivation but they ignore to the extent to which this is the case 
and how testing can be made to function as a valid instructional tool. 

3.3 Difficulties in Methodology’ for Oral Tasks 

In addition to having a limited view of assessment and examination, teachers of English in Spain also tend not to use 
English extensively in the classroom. In fact, in a recent survey among second year students at the Polytechnic 
University of Valencia, 30 second year students in the Travel & Tourism degree program responded to a 
questionnaire in which they were asked, among other things, about the language of instruction in their high school 
English courses; 19 of them (63%) responded that Spanish had been used most of the time while only 4 responded 
that most of their classroom interactions had been in English. Therefore, it can be concluded that the policies and 
methods used in the classroom will have to be revised in addition to the intention to introduce a computer-based 
examination. 

Among our findings after the training sessions we first found that some of the teachers in the study had originally 
been French teachers and that they still had great limitations in English. These teachers had had to change to 
teaching English (and to learn it!) because French is no longer used as a second language in most high schools. 
Some of the other problems we came across were the following: 

1) The teachers limit their focus to preparing students for the P.A.U. to the aims and scope of preparation for the 
P.A.U. Since the communicative skills of speaking and listening are not part of the test, the teachers feel that they 
should devote their time to activities more productive for the exam while ignoring aspects such as thinking and 
language use and knowledge, which are irrelevant to the exam. 

2) The teachers have given up. Many of the teachers acknowledged that they would like to devote more time to 
learning activities related to oral development but that the importance of the test is such that the students must 
master the written skills. 

3) The teachers complain about their circumstances. The teachers in the study tended to excuse their behavior 
because their colleagues in the primary schools do not teach speaking either. They claimed that, by the time their 
high schoolers reach the last year when they have them as students, it is just too late to teach them speaking. 

3.4 Constraints in Technology Use 

Although the training course for this study had been advertized with the name “Using Technology for ELT,” we 
received some surprising reactions to it. First, some of the teachers who attended the training sessions reported that 
they had been “forced” by their own schools to attend them. Second, one of the teachers responded in the initial 
questionnaire that she “hated computers”. However, we found two kinds of more extensive problems: basic use of 
technology on the one hand and attitudes towards technology on the other. The former was handled through 
intensive training during the course, but we noted that the difficulty does not reside in the use of certain computer 
software programs but in the teachers’ attitudes towards implementing technology in their teaching, the latter 
problem. It was observed in the training sessions that the significant difficulty does not lie in how to teach but in 
what to teach. Nevertheless, when the teachers were properly instructed in the new technologies, content became the 
key matter in language instruction. For instance, after 12 hours of instruction in a computer testing platform, the 
teachers felt more confident and, at least in the training role plays, they paid more attention to what was to be taught 
than to how to teach their audience to use or perform a task on the given platform. 

4. Implications for Teacher Training 

Obviously, teacher training programs need to show the usefulness and applicability of what is learned in sessions. In 
this sense teachers must be helped to manage effectively the applications that will be used in their actual classes 
instead of being provided a general approach to ICT. If this is done, it will make the training period meaningful and 
worthwhile. The following issues should also be considered: 
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1) Teachers should be helped to understand the tools that they will have to use and their role in the test. This means 
that teachers must be shown that computers can be used to obtain evidence of areas that have not been measured to 
date (i.e. the oral skills). 

2) Fears natural to people not familiar with digital technology must be eradicated. Thus, teachers need to be helped 
to understand their own limitations and to overcome them through team teaching, support from their colleagues, and 
cooperation with the most skillful students. 

3) Teachers must be made aware that the ongoing effort to stay up-to-date on technological advances is worthwhile 
and highly accessible after an initial stage, which could be called a “discovery experience”. 

4) Teachers must be informed of the existence of forums and sites on the Internet where they can share and learn 
from the other teachers’ experiences. Examples are Dave’s ESL Cafe, TEFL NET, Humanising Language teaching: 
http://www.hltmag.co.uk/ or www.isabelperez.com, but many more can also be found. 

5. Conclusions 

Overall, it should be pointed out that technology is definitely only one of the very many factors that have an 
incidence on teacher training in CALT. While trainers do need to make technology accessible to the teacher trainees, 
trainers also need to emphasize the contents and tasks that will be measured (for example, the oral skills that can be 
assessed through computer technology) (Pan, 2009) as well as how to do so (Inbar-Lourie & Donitsa-Schmidt, 
2009). Opportunities for further development should also be promoted alongside the possibility of teachers’ future 
collaboration. In the case of this study the participants kept in touch with each other after the course and worked 
together on certain school projects. In addition, they learned to use the technology beyond the P.A.U. test. In general, 
teacher training courses can be difficult to run but an awareness of participants’ potential fears and problems can 
facilitate the trainer’s role. In this sense, this paper is only a first approach to a complex matter which has received 
very limited attention in the professional literature to date. 

In relation to methodology, teachers should be aware of times when washback is not absolutely necessary. 
Institutions should also ask their students about their needs for washback because test-oriented courses may be 
focused differently from others according to circumstances (some courses may not demand a strong emphasis on 
biasing instruction) (Qi, 2005; Scott, 2007; Gao, 2009). Minimally students have “different constructions of the 
themselves as test-takers, the test, and their performance in the test” and their accounts will include their 
“expectations and explanations for success or failure as well as credit or blame” (Huhta, 2006: 326). At the same 
time, it should be noted that washback requires strong research support based on baseline studies (Wall & Horak, 
2007) to support not only school practices but also language policies as observed in current tests (Shohamy, 2007). 

Note 

We would like to express our gratitude to Spain’s Ministry of Education for supporting and funding this study under 
the PAULEX project (HUM2007-66479-C02-01/FILO). We would also like to acknowledge joint funding from the 
Consejeria de Educacion of the Madrid Autonomous Region and the University of Alcala’s R&D program for new 
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